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METHOD AND SYSTEM FOR PROVIDING PERSONALIZED 
BROADCAST TELEVISION HIGHLIGHTS SERVICE 

Field of Invention 

The present invention relates generally to audio/visual content presentation, and 
particularly to methods and systems for presenting high quality audio/visual content 
and user-interactive information. 

Background of Invention 

The rise in popularity of the global interconnection of computing devices and 
networks commonly referred to as the Internet offers new levels of interactivity for 
users accessing entertainment and information. However, Internet communication 
channels are relatively low in bandwidth as compared to more traditional 
entertainment transport systems such as broadcast television (TV). Further, the 
amount of bandwidth available is typically inconsistent over time. Because high- 
quality video, such as broadcast TV quality, requires a consistently available, 
relatively high bandwidth, audio/visual content provided to users over the Internet is 
typically of low-quality, with regard to frame-rate and resolution, or undesirably 
slow to download or otherwise access. 

In contrast, broadcast television in its current form offers better quality 
audio/visual content, and is likely to improve in the near future with the widespread 
arrival and acceptance of digital television (DTV) and high-definition television 
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(HDTV). However, this next generation television still doesn't offer the interactivity 
and rich variety of user-interactive content that the Internet does. A characteristic 
that broadcast television and the Internet do have in common is the skyrocketing 
amount of available content, leading to "information overload" for the user. Smart 
systems are needed to help a user sort through this vast amount of content. 

Accordingly, it is an object of the present invention to provide a system and 
method for providing delivery of high quality video in combination with interactive 
components designed to help the user sort through, or have automatically sorted 
through, available content and further retrieve user-interactive content. 

Personalized television services such as TiVo and REPLAYTV provide 
personalized, automated digital recording of broadcast video. However, while these 
services address some of the shortcomings addressed above, such as the problem of 
finding what you want in the glut of broadcast video content, they are only crudely 
personalized. Essentially, they are user-friendly, minimally intelligent, digital video 
recorders. WEBTV offers services that begin to integrate TV and the Internet, 
however does not offer program-specific metadata. 

In the area of sports, PRAJA offers sports-program-content-based 
information over the web, with metadata and hyperlinks. However PRAJA does not 
integrate broadcast TV in any way, and thus can only offer Internet-quality 
audio/visual content. Also, PRAJA does not offer any personalization of the 
information presented. 
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Summanr of Invention 

A method for identifying at least a portion of high quality audio/visual 
content being of potential interest to a user and available via a first communications 
network and corresponding user-interactive content available via a second 
communications network, the method including: storing data indicative of at least 
one user interest; receiving data indicative of scheduling information for the high 
quality audio/visual content via the first or second communications network; 
receiving data indicative of the high quality audio/visual data and data indicative of 
the user-interactive content via the first or second network; and, automatically 
identifying select portions of the high quality audio/visual content and corresponding 
user-interactive content dependently upon the stored data indicative of the at least 
one user interest using the data indicative of scheduling information, data indicative 
of the high quality audio/visual data and data indicative of the user-interactive 
content. 

Brief Description of the Figures 

Figure 1 illustrates a block diagram of a system according to one aspect of 
the present invention; and, 

Figure 2 illustrates a block diagram of a set top box according to another 
aspect of the present invention and suitable for use with the system of Figure 1 . 
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Detailed Des cription of the Invention 

The present invention introduces a hybrid system that merges the advantages 
of broadcast TV and the Internet, which is hereinafter referred to as a Personalized 
Broadcast TV Highlights Service (PBTV). According to one aspect of the present 
invention, PBTV delivers broadcast-quality audio/visual content plus web content 
and interactivity to provide a personalized and enriched viewing experience. Based 
on an individual user's preferences, the PBTV system provides efficient storage, 
indexing, search, and retrieval of broadcast-quality digital audio/visual content, 
along with related multimedia content, and hyperlinks to related Web Sites. 

Referring now to the Figures, like references identify like elements of the 
Invention. Figure 1 illustrates a block diagram of a system 10 according to one 
aspect of the present invention. The system 10 includes at least one consumer 
electronics, set-top box (WEB/DVR/STB) or STB 100, at least one website 200 
(PBTV website 200), at least one content provider (NCP) 300 and at least one digital 
video broadcaster 400. The present invention will be discussed in a non-limiting 
example which includes a single user 100, website 200, NCP 300 and broadcaster 
400 for sake of clarity. The Digital Video Broadcaster 400 provides high-quality 
digital audio/visual (A/V) content 410 in the PBTV system 10. This A/V content 
410 can be provided via a satellite or cable television system for example. 
Participating NewMedia Content Providers (NCPs) 300 provide metadata and 
multimedia information and analysis 310, as well as hyperlinks to web sites related 
to high quality audio/visual content 410. Metadata, as used herein generally refers to 
data which describes or is associated with programs 410 or portions of programs 
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410. The Web/Digital-video-recorder/Set-top-box STB 100 is a consumer 
electronics device. The PBTV Web Site 200 acts as a source or gateway for 
electronic program guides (EPGs), and as a gateway to NewMedia Content 
Providers (NCPs) 300 for the STB 100. Program-related metadata and multimedia 
content, e.g., program-related information and analysis, is downloaded 210 via the 
PBTV Web site 200 to the user's STB 100 in response to requests therefore as will 
be described. 

Referring now also to Figure 2, therein is illustrated a block diagram of an 
STB 100 according to another aspect of the present invention and suitable for use 
with the system of Figure 1. The STB 100 includes a microprocessor 120, decoder 
1 10 for decoding broadcast content, modem 130 for communicating with the website 
200, e.g. communications 180, 210, and accessing the Internet, and storage device 
140 such as a hard disk drive, for storing A/V data and other information. 
Communications mediums for the back-channel to the website 200 from the STB 
100 include conventional telephone lines, digital subscriber lines (DSLs), cable 
modem channels and cellular and other wireless networks for example. The decoder 
110 also provides audio/visual content to the display device 160, such as a television 
or other suitable monitor. The device 100 further includes one or more input devices 
170 such a keyboard, mouse or remote control, for operation of the STB 100 using a 
Graphical User Interface (GUI) displayed using the display device 160 via the 
microprocessor 120 and decoder 110. 

Video programming content 410 is provided by the digital video broadcaster 
400 to the STB 100 and NCP 300, and is the actual broadcast-quality video the 
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PBTV system 10 delivers to the display 160. The Electronic Program Guides 
(EPGs) are provided by either the broadcaster 400 or via the PBTV Web Site 200, 
and provide scheduling information for programming content. Metadata is supplied 
by an NCP 300 via the PBTV Web Site 200, and describes the program content 410, 

5 including time-of-occurrence information, such as for a scene beginning at time "X", 

data describing who, what, where, when, why, or how. User Interest Profiles (UIPs) 
are personalized by a user via the PBTV Web Site 200 and describe, for each 
participating NCP 300, the interests of an individual user, expressed in terms of the 
metadata available from that NCP 300. For example, "I am interested in all scenes 

10 where Brad Pitt appeared," or "I am interested in all New York Jets third-down plays 

where Vinny Testaverde passed the ball" can be used to generate UIPs. UIPs can be 
stored as database records. Related multimedia content is information and analysis 
furnished by the NCPs 300 via the PBTV Web Site 200, and includes such things as 
related news, e.g. an article about Brad Pitt's previous movie or Vinny Testaverde's 

15 injury status, evaluation of the program content, e.g. a criticism of Brad Pitt's acting 

or Vinny Testaverde's game performance, or related statistics, e.g. Brad Pitt's 
resume or third-down statistics for the Jets for the game and the season. Hyperlinks 
to related multimedia content, e.g. websites, are also furnished by an NCP 300 via 
the PBTV Web Site 200, and provide a user with opportunities for further 

20 exploration, e.g. a hyperlink to a Universal Pictures web site or to an NFL web site. 

Referring still to Figure 2, a PBTV user begins by personalizing a UIP 
template for a NewMedia Content Provider (NCP) 300 associated with one of the 
user's general areas of interest, e.g. sports, movies or news, for example as 
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communication 180. The UIP template for an NCP 300 is preferably designed to 
facilitate searching and retrieving video and related content using the metadata 
available from that NCP 300, and the EPG. In other words, a personalized UIP for 
an NCP 300 describes the user's personal interests in terms of the metadata available 
from that particular NCP 300. The EPG data and metadata can be queried using a 
UIP to not only identify programs 410 of potential interest to a user, but portions of 
programs 410 of potential interest to a user as well. 

Based on the UIP and the EPG, the PBTV system 10 identifies, by using 
appropriately formed queries based upon the UIP, which broadcast programs 410 are 
of potential interest to the user, and the times those "interesting" programs 410, or 
even portions of programs 410 will be broadcast. According to an aspect of the 
invention, based upon the results, the STB 100 automatically records the compressed 
bitstreams of those "interesting" programs 410, or portions of programs 410, using 
the storage device 140. The user can play back the PBTV-recorded broadcast- 
quality video programming, thus seeing programming content 410 that has been 
sifted according to the user's UIP. According to another aspect of the invention, this 
playback also facilitates user-guided exploring of related multimedia content and 
hyperlinks provided by an NCP 300 via the PBTV Web Site 200 by means of a GUI. 
The GUI used to present the PBTV content for a particular NCP 300 is preferably 
tailored to the needs of the metadata and related content available from that NCP 
300. This can also be accomplished by analogously querying data indicative of 
websites using a UIP, to idehtify ones of the websites of potential interest to the user. 
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According to an aspect of the present invention, in order to use the EPG and 
metadata to drive the STB 100 to accurately record and index the desired broadcast 
video, and then retrieve that stored video, the PBTV system 10 uniquely identifies 
any arbitrary point in the video broadcast, and links that instant in the broadcast with 
corresponding points in the metadata and the EPG. According to another aspect of 
the present invention, the PBTV system 10 achieves this synchronization using the 
Presentation Time Stamp (PTS) of the programming content 410, which can take the 
form of an MPEG-2 ATSC compliant video bitstream, for example. By calculating 
a difference between the PTS at the beginning of a broadcast program, which 
corresponds to a time in the EPG associated with this program, and the PTS at a 
particular point in the broadcast program, that point in the program can be uniquely 
identified within a resolution of the frequency-of-occurrence of the PTSs. Therefore, 
by simply receiving the programming content 410, decoding the PTSs, and 
calculating changes in the PTSs, a metadata author such as NCP 300 can uniquely 
identify or refer to any event in the broadcast programming content 410. Since a 
PTS occurs at least every 700 milliseconds in an ATSC compliant bitstream, this 
PTS-based approach to synchronization permits an event in the broadcast 
programming content 410 to be identified to within 21 frames, or less than 1 second. 
The recording cues calculated for the Web/DVR/STB 100 recording process based 
on the EPG and UIP can take this tolerance into account, to insure that all desired 
audio/visual content is recorded. 

As set forth, the PBTV system 10 allows a user to specify not only which 
programs are of interest, but even which parts of those programs are of interest using 
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UIPs. According to yet another aspect of the invention, in cases where only portions 
of a program 410 are of interest, in order to more efficiently utilize the capacity of 
the storage device 140, the STB 100 uses a video "crunching" operation to further 
compress the recorded bitstream. According to yet another aspect of the present 
invention, where metadata cannot be guaranteed to be available concurrently with 
programming content 410, in the case of a live sports broadcast for example, the 
STB 100 begins by recording in its entirety, a program 410 which, based on 
information in the EPG and the UIP, may be of interest to the user. Once 
corresponding metadata is available from an NCP 300 via the PBTV Web Site 200 
for a program 410 that has been recorded, the system 10 queries the now available 
metadata based upon the UIP, and "crunches" the previously recorded bitstream by 
removing from the STB 100 storage device 140 any "uninteresting" portions of the 
bitstream, such as advertisements for example. The exact portions of the previously 
stored bitstream to be erased are determined by matching the PTS information in the 
metadata with the PTSs in the encoded video bitstream, as has been discussed This 
approach of initially recording an entire program and then later "crunching" that 
program, permits a time lag between when a program 410 is broadcast, and when the 
program-related metadata is available. According to yet another aspect of the 
present invention, where metadata cannot be guaranteed to be available concurrently 
with programming content 410, delivery of the programming content 410 to the 
display device 160 can be buffered or delayed, using the STB for example, for a 
suitable temporal duration, such as 5 seconds for example, so as to allow an 
opportunity for the metadata to become available. 
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According to another aspect of the present invention, another approach to 
video "crunching" is graduated removal. With this technique, rather than completely 
removing "uninteresting" portions of the stored, compressed bitstream, these 
portions are transcoded to a lower bitrate, so that they take up less space in storage 
device 140. User prioritization on video content, expressed in the UIP, can act as a 
guide for this graduated removal process. Graduated removal can also be utilized 
when the available capacity of storage device 140 approaches a given threshold, in 
order to free-up space so that additional compressed audio/visual content can be 
recorded for example. 

The recording followed by "crunching" of a stored video can be performed 
either serially or in a pipelined fashion, depending on the amount of lag time 
between program broadcast 410 and metadata availability from the NCP 300. 
Following the video crunching process, in order to more efficiently use available 
storage space of the device 140, de-fragmentation as is conventionally understood 
can be is performed on the remaining compressed video bitstreams so as to achieve 
more efficient packing of the "crunched" programming content. 

UIPs can reside in either the user's STB 100 or PBTV Web Site 200. The 
EPG can be obtained as part of the digital video broadcast 410, or from the PBTV 
Web Site 200, and can be stored at the PBTV Web Site 200, or in the user's STB 
100. The computations to control automated recording can be performed at the 
PBTV Web Site 200, or in the user's STB 100. Further, appropriately sifted 
metadata can be batch downloaded to the user's STB 100 or reside at the PBTV Web 
Site 200 and be accessed on an as needed basis by the STB 100. Related multimedia 
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content can also be pre-downloaded to the user's STB 100, or downloaded on-the-fly 
at time of video playback from the web site 200. 

Although the invention has been described and pictured in a preferred form 
with a certain degree of particularity, it is understood that the present disclosure of 
the preferred form, has been made only by way of example, and that numerous 
changes in the details of construction and combination and arrangement of parts may 
be made without departing from the spirit and scope of the invention as hereinafter 
claimed. It is intended that the patent shall cover by suitable expression in the 
appended claims, whatever features of patentable novelty exist in the invention 
disclosed. 
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Claims 
We Claim: 

1. A system (10) for selectively presenting portions of at least broadcast quality 
audio/visual content (410) from at least one broadcaster via a broadcast network and 
portions of user-interactive content (310) accessible via a computer network based 
upon a user's interests and using a presentation device (160), said system (10) 
comprising: 

an input device (170) for providing data indicative of at least one user 
interest; 

at least one web site (200) for providing access to data (210) indicative of 
said audio/visual content; 

a plurality of memory locations storing computing instructions for identifying 
said portions of said audio/visual content using said data indicative of said at least 
one user interest and data indicative of said audio/visual content; and, 

a storage device (140) for recording said identified portions of said 
audio/visual content. 

2. The system of Claim 1, wherein said instructions utilize time stamps (PTS) 
to identify said portions of said audio/visual content (410). 
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3. The system of Claim 1, further comprising a set top box (100) which 
includes said storage device (140) and a processor (120) for executing said 
instructions, and said plurality of memory locations. 

4. The system of Claim 3, wherein said storage device (140) records said at 
least one other portion than said select portion of said audio/visual content, and said 
processor (120) and storage device (140) erase said at least one other portion based 
upon said data indicative of said audio/visual content. 

5. A method for identifying at least a portion of high quality audio/visual 
content being of potential interest to a user and available via a communications 
network, said method comprising: 

storing data indicative of at least one user interest; 

receiving data indicative of scheduling information for said audio/visual 
content; 

receiving data indicative of said audio/visual content; and, 

automatically identifying select portions of said audio/visual content 
dependency upon said stored data indicative of said at least one user interest using 
said data indicative of scheduling information and data indicative of said 
audio/visual content. 
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6. The method of Claim 5, further comprising: 

receiving data indicative of user-interactive content corresponding to said 
audio/visual content and being available via a second network; and, 

automatically identifying select portions of said user-interactive content 
using said data indicative of at least one user interest and data indicative of said user- 
interactive content; 

wherein said first network comprises a broadcast network and said second 
network comprises a computer network. 

7. The method of Claim 5, further comprising: storing at least a portion of said 
audio/visual content so as to be accessible to said user at a rate sufficient to enable 
real-time presentation of said audio/visual content. 

8. The method of Claim 7, further comprising selectively deleting at least a 
portion of said stored audio/visual content. 

9. A device (100) for selectively presenting select portions of audio/visual 
broadcasts (410) received via a broadcast network (400) and being of potential 
interest to a user, said device (100) comprising: 
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at least one input (170) for receiving data indicative of at least one user 
interest; 

a decoder (110) for receiving said audio/visual broadcasts (410); 

a modem (130) for receiving data associated with said audio/visual 
broadcasts (410) via a computer network (200); 

at least one memory device (140) for storing at least said select portions of 
said audio/visual broadcasts (410); and, 

at least one processor (120) electrically coupled to said decoder (110), 
modem (130) and memory device (140), and for identifying said select portions of 
said audio/visual broadcasts (410) using said received data associated with said 
audio/visual broadcasts (410). 

10. The device of Claim 5, further comprising at least one processor memory 
storing instructions for causing said processor (120) to selectively delete fractions of 
said stored audio/visual broadcasts (410) from said at least one storage device (140) 
using said received data indicative of said audio/visual broadcasts (410). 
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